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Abstract 

Background: In the context of the massive scale up of malaria interventions, there is increasing recognition that 
the current capacity of routine malaria surveillance conducted in most African countries through integrated health 
management information systems is inadequate. The timeliness of reporting to higher levels of the health system 
through health management information systems is often too slow for rapid action on focal infectious diseases 
such as malaria. The purpose of this paper is to: 1) describe the implementation of a malaria sentinel surveillance 
system in Ethiopia to help fill this gap; 2) describe data use for epidemic detection and response as well as 
programmatic decision making; and 3) discuss lessons learned in the context of creating and running this system. 

Case description: As part of a comprehensive strategy to monitor malaria trends in Oromia Regional State, 
Ethiopia, a system often malaria sentinel sites was established to collect data on key malaria morbidity and 
mortality indicators. To ensure the sentinel surveillance system provides timely, actionable data, the sentinel facilities 
send aggregate data weekly through short message service (SMS) to a central database server. Bland-Altman plots 
and Poisson regression models were used to investigate concordance of malaria indicator reports and malaria 
trends over time, respectively. 

Discussion: This paper describes three implementation challenges that impacted system performance in terms 
of: 1) ensuring a timely and accurate data reporting process; 2) capturing complete and accurate patient-level data; 
and 3) expanding the usefulness and generalizability of the system's data to monitor progress towards the national 
malaria control goals of reducing malaria deaths and eventual elimination of transmission. 

Conclusions: The use of SMS for reporting surveillance data was identified as a promising practice for accurately 
tracking malaria trends in Oromia. The rapid spread of this technology across Africa offers promising opportunities 
to collect and disseminate surveillance data in a timely way. High quality malaria surveillance in Ethiopia remains a 
resource intensive activity and extending the generalizability of sentinel surveillance findings to other contexts 
remains a major limitation of these strategies. 
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Background 

Disease surveillance is one of the fundamental functions 
of public health systems [1], and recording and reporting 
of malaria cases is practiced in nearly all countries where 
transmission persists. Malaria remains a reportable dis- 
ease even in countries, such as the United States of 
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America, where sustained autochthonous transmission 
has long been essentially absent [2]. To realistically em- 
bark on the road towards malaria elimination, timely 
provision of accurate malaria surveillance data is neces- 
sary [3] . In the context of the massive scale up of malaria 
interventions, there is increasing recognition that the 
current capacity of routine malaria surveillance, con- 
ducted in most African countries through integrated 
health management information systems (HMIS), is in- 
adequate: indicators are poorly defined; data reporting 
and completeness are often of unknown quality; and 
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timeliness can be extremely variable. Further, there is 
often limited capacity for data analysis, data interpre- 
tation, and action. Additionally, in many locations 
confirmatory malaria diagnosis is of variable quality or 
absent [4,5]. 

In most sub-Saharan African (SSA) countries, routine 
malaria surveillance involves collecting and reporting 
aggregate data from public health facilities through the 
national HMIS. Data are collected about patients pre- 
senting at public facilities with symptoms of disease, and 
malaria cases are reported based on the diagnostic ser- 
vices available at the facility, or sometimes reported on 
clinical signs and symptoms only when diagnostic tools 
are absent. Where laboratory services are available, mi- 
croscopy has generally been the most common form of 
confirmatory diagnosis. Although microscopy is gener- 
ally considered the gold standard for malaria laboratory 
diagnosis, rapid diagnostic tests (RDT) have recently 
been rolled out in many SSA settings, particularly at the 
community level where microscopy is generally not 
available. This expansion may help to rationalize treat- 
ment practices and to improve the quality of surveillance 
data in many locations [6,7]. Data about malaria ill- 
nesses, with or without laboratory diagnostic confirm- 
ation, tend to be compiled weekly or monthly at each 
health facility and then reported up a vertical chain, with 
further aggregation at each level in the health system 
until reaching the most central level, such as the HMIS 
surveillance headquarters. These aggregated data are 
then ideally used by National Malaria Control Pro- 
grammes (NMCP) and Ministries of Health (MoH) to 
make decisions related to the timing and frequency of 
prevention, treatment, and control interventions. 

In general, routine surveillance systems tend to miss 
many malaria cases because they are either treated out- 
side the formal public health system, not treated at all; 
underreporting and failure to capture testing and treat- 
ment data within the system can also lead to missed 
malaria cases. Concurrently, routine surveillance data 
may vastly overestimate the burden of malaria within 
the public health system due to under-utilization or lack 
of diagnostics, and a consequent reliance on clinical 
diagnosis. Additional challenges, such as poor data re- 
cording practices and lack of supervision, have an un- 
known effect on routine system data quality. In some 
settings, failure of facilities to completely report upwards 
in the reporting chain also results in aggregation of in- 
complete datasets and generalized under-reporting of 
malaria burden. Additionally, the timeliness of reporting 
to higher levels of the health system is often too slow for 
rapid action on focal infectious diseases, such as malaria. 
Validation of such data is nearly impossible in the ab- 
sence of gold standard data sources to assess the sys- 
tem's sensitivity and diagnostic quality control, which is 



often logistically difficult for microscopy and currently 
unavailable for RDTs. Furthermore, support and supervi- 
sion for surveillance activities is often lacking within the 
HMIS. Small-scale sentinel surveillance with enhanced 
supervision and rapid reporting mechanisms are a viable 
alternative to relying solely on data collected through 
the country's routine HMIS and may provide the best 
available gold standard for malaria surveillance and epi- 
demic detection. 

Ethiopia's national malaria control strategic plan in- 
cludes goals to eliminate malaria in low-transmission 
areas and achieve near zero deaths due to malaria by 
2015. To monitor progress towards these goals, a system 
for capturing both local and regional transmission is es- 
sential. Particularly, there is a need for an effective sur- 
veillance system that can target focal areas of infection, 
increase capacity to identify transmission hot spots, and 
monitor near real-time malaria data to rapidly identify 
changes in malaria transmission, morbidity and mortal- 
ity. The purpose of this paper is to: 1) describe the 
design and implementation of a malaria sentinel surveil- 
lance system in Oromia, Ethiopia; 2) describe data use 
for epidemic detection and response, as well as program- 
matic decision making; and 3) discuss lessons learned re- 
lated to creating and running this system to provide 
practical examples and suggestions for improvement and 
use in other systems or settings. 

Case description 

As part of a comprehensive strategy to monitor malaria 
trends in Oromia Regional State, Ethiopia, a system of ten 
malaria sentinel sites was initially established in early 2010 
to collect data on key malaria morbidity and mortality 
indicators. This work was supported by the U.S. President's 
Malaria Initiative (PMI) and implemented by Tulane Uni- 
versity and Addis Continental Institute of Public Health in 
close collaboration with a range of stakeholders, including 
the Federal Ministry of Public Health (FMOH), Oromia 
Regional Health Bureau (ORHB), and the Ethiopian Health 
Nutrition and Research Institute (EHNRI). This work was 
approved by the Biomedical Institutional Review Board of 
Tulane University and the Ethical Review Committee of 
the Addis Continental Institute of Public Health. 

The strengths and weaknesses of the system were 
assessed by a combination of methods over three years 
of implementation using various tools including: needs 
assessments, stakeholder interviews, supervision reports, 
data quality audits, laboratory and system quality control 
assessments, and analysis of surveillance data. These 
data formed the framework for assessing the lessons 
learned and best practices presented below. Methods for 
data quality audits and the general framework for assess- 
ment were based on frameworks outlined in U.S. Cen- 
tres for Disease Control (CDC) guidelines [8,9]. 
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Malaria situation in Ethiopia 

In Ethiopia, altitude and climate are the most important 
determinants of malaria transmission, which is highly 
seasonal and predominantly unstable [10]. The peak 
malaria transmission season occurs between September- 
December, while a minor transmission season occurs in 
April-May. There are four major eco-epidemiological 
malaria transmission strata in Ethiopia: 1) malaria-free 
highland areas above 2,500-meter altitude; 2) highland 
fringe areas between 1,500 and 2,500 meters (affected by 
frequent epidemics); 3) lowland areas below 1,500 me- 
tres (seasonal pattern of transmission); and 4) stable 
malaria areas (year-round transmission; limited to the 
western lowlands and river basins) [11]. Plasmodium fal- 
ciparum and Plasmodium vivax are the dominant mal- 
aria parasites in Ethiopia, with their relative contribution 
thought to be approximately 60% and 40% of all malaria 
cases, respectively. Plasmodium malariae accounts for 
less than 1% of cases and Plasmodium ovale is very rare 
[12-14]. The primary malaria vector in Ethiopia is 
Anopheles arabiensis; secondary vectors include Anoph- 
eles pharoensis, Anopheles funestus and Anopheles nili 
[11,15-18]. Since little malaria transmission is apparent 
at altitudes above 2,000 metres, most malaria interven- 
tions, including insecticide-treated nets, are targeted to 
areas below 2,000 metres [11]. 

Selection of sentinel surveillance sites 

Ten primary health care units (PHCU) in Oromia were 
initially selected as sentinel sites for malaria surveillance 
activities. PHCU serve a catchment area of -25,000 
people and consist of district (woreda) -level health cen- 
tres and satellite community (kebele) -level health posts. 
Health centres are primarily concentrated in urban 
areas, and are usually staffed by at least one health offi- 
cer, laboratory technicians, pharmacists or druggists, and 
midwives. Most health centres have inpatient capabil- 
ities, albeit limited, and as such they are the first referral 
point for severe malaria cases identified at health posts. 
Health posts are usually in rural areas and focus on pre- 
ventive services, providing only limited curative services, 
mainly for malaria. Health posts are staffed by two 
health extension workers (HEWs), who are fully-salaried 
FMOH staff. These HEWs are primarily women with a 
high school diploma and typically originate in the com- 
munities they serve. HEWs provide service delivery on 
16 selected health packages for which they receive spe- 
cific training, including malaria. They are expected to 
confirm all suspected malaria cases (fever cases) with a 
multi-species RDT and provide appropriate treatment 
based on RDT results. 

Following a national stakeholder meeting, the follow- 
ing criteria were used for selecting PHCUs, to serve as 
sentinel sites: 1) presence of an outpatient clinic that 



sees an average of at least 50 patients per day; 2) labora- 
tory capacity to diagnose malaria using microscopy; 3) 
ability to provide artemisinin combination therapy (ACT) 
as first-line treatment for uncomplicated malaria during 
selection visit; 4) pre-existing designated personnel re- 
sponsible for data collection and reporting at the facility 
during selection visit; 5) situated below 2,000 metres above 
sea level in a malaria transmission area; and 6) available 
electricity and year-round access via road. A needs assess- 
ment was conducted at 20 health centres which met the 
above criteria prior to final consideration and selection. 
Health centres meeting the above criteria were considered 
by the ORHB; community acceptance and political com- 
mitment were other important final criteria for selection. 
Efforts were also made to include health centres in areas 
where the timing, duration, and intensity of transmission 
varied (e.g. more than 6 months transmission and less 
than three months transmission; less than one confirmed 
case/500 persons per year and more than one confirmed 
case/500 persons per year; predominance of P. falciparum 
or P. vivax), and where both microscopy and RDTs were 
used. Health centres formed the initial and central points 
for each sentinel surveillance site; over time expansion to 
all satellite health posts within the 10 PHCUs was com- 
pleted. Full expansion of the malaria sentinel surveillance 
system was achieved in 2012 (after approximately 2 years), 
it included 10 health centres and their 73 satellite health 
posts (Figure 1). 

Data collection and reporting 

Patient data were collected at two main service delivery 
points: health centres and health posts. At health centres, 
an outpatient department (OPD) register and a laboratory 
register captured patient demographic and malaria data. 
The OPD registers were designed to capture information 
on patient age, location of residence, fever history, labora- 
tory tests requested, laboratory results for malaria and 
relapsing fever through microscopic examination of 
Giemsa or Field stained blood slides or multi-species RDT 
(CareStarf, AccessBio, NJ), species-specific final diagnosis 
(i.e., uncomplicated malaria, severe [complicated] malaria, 
other), drugs prescribed, inpatient admittance, death and 
referral to higher level facilities. At health posts, data were 
collected from the routine fever and malaria patient regis- 
ter. Because HEWs at health posts can currently only pro- 
vide treatment for one acute illness (malaria infection), 
one patient register covers all diagnostic testing and treat- 
ment services. 

Surveillance field support staff initially visited each 
health centre every two weeks (after one year this was 
changed to monthly) to work with health centre staff to 
extract relevant malaria data from the registries and 
share findings from previous months and from other fa- 
cilities in the surveillance network. Data in the field were 
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collected using paper forms; after each visit the field 
support staff entered all data into a pre-populated data 
base IV format (dbflV) file using Epilnfo 7.0 (U.S. Cen- 
ters for Disease Control and Prevention). Additional 
points of communication and data collection at health fa- 
cilities included the medical stores and dispensaries, where 
stocks of malaria-related commodities are tracked. Health 
posts were visited monthly to extract relevant malaria data 
from paper registers. In addition to register abstraction, all 
health posts and health centres sent data electronically in 
aggregate on a weekly basis via short message systems 
(SMS). Data from the SMS system are maintained in a 
separate database, as paper registry and laboratory records 
were considered gold standard data. These data were com- 
pared using methods discussed below to ensure accuracy 
of rapid reporting data. 

To ensure the sentinel surveillance system provides 
timely, actionable data, the sentinel facilities send weekly 
SMS data messages to a central server. These messages 
provide essential malaria data to notify health officials of 
potential case build-ups requiring a targeted response; 
the paper data provide more detailed data for in-depth 
analysis. The main system components of the SMS- 



based data collection are a multi-channel communica- 
tion processor (MMP) and a management and reporting 
portal (MRP). The MMP is the component used to man- 
age channel communications, such as SMS. The MRP is 
browser-based and interfaces with various desktop appli- 
cations such as Microsoft Excel or Adobe Reader to 
allow for reports to be exported in various formats. The 
entire system interfaces with the Ethiopian Telecommu- 
nications Corporation to facilitate communication via 
SMS and the internet. There are three general system 
users of the data: 1) data submitters (e.g., HEW at health 
posts and laboratory technicians at health centres); 2) 
data users (e.g., stakeholders, data managers at project 
institutions, health workers); and 3) administrators (e.g., 
field support team members at the implementing part- 
ner institution). Figure 2 presents a mobile phone screen 
shot to illustrate the data reporting format; it also lists 
the indicators captured as part of the systems' weekly 
malaria reports. All data in monthly reports are pre- 
sented in tabular and graphic formats. 

The web-based interface allows users to view the data 
in near real-time for assessment of expected and actual 
malaria cases occurring within the defined area. A 
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[vvmd] [dale] [user id] [facility id] [type] [patients] [tested] [Pf+] [Pv+] [mixed] [negate] 



Where 



Wmd = sms keyword 

Date -report date (DD/MMj 

User id = data submder unique id number 

Facility id = health facility ID 

Type = adult (a) or child (cj 

Patients = total number of patients seen 

Te sted - total nu mbe r of patients te ste d 

Pf+ ■ total number of positive tests for Pf 

Pv+ = total number of positive tests for 

Mixed = total number of mixed cases 

Negative - total number of negative tests 




To 0911514590 

wmd 1012 023 
145 a 56 40 5 8 

10 17 





Figure 2 Screen shot of mobile phone displaying data reporting format used in Ethiopia at the health post level. 



second mechanism of epidemic notification occurs in 
the form of both an email and SMS text sent to the 
users when a specific threshold is reached. For example, 
when a data report is submitted, the system automatic- 
ally checks the values of P. falciparum and P. vivax test 
positivity rates (TPR) as well as numbers of laboratory- 
confirmed cases against predefined threshold values for 
that health facility. If any of the indicators meet or ex- 
ceed the threshold value for that facility in a given week, 
the system automatically sends an e-mail and text to the 
health centre or post, or to other designated users where 
the epidemic notification had been triggered. The mes- 
sage includes the indicator that triggered the alert, the 
current data value, the threshold value, and the catch- 
ment area to which the facility is associated. 

The system interacts with the standard surveillance 
system for malaria in Ethiopia at multiple points. These 
include notification of relevant officials at all levels, in- 
cluding woreda malaria focal persons, of unusual malaria 
morbidity or mortality reports, in-person and telephone 
follow-up at the woreda and facility level for routine 
supervision and the communication of results of surveil- 
lance activities to the higher levels of the Ethiopian 
health system including the Federal Ministry of Health. 

Data quality assurance 

To reduce the potential for SMS system error, several 
automatic basic data logic checks are employed on re- 
ports. These include checks to ensure that confirmed 
cases do not exceed malaria tests conducted and other 
similar logical checks. If an SMS data report fails to 
meet these checks, the surveillance system sends a mes- 
sage to the sender indicating a resubmission is necessary. 
The system overwrites the original report once a resub- 
mission has been submitted, and a message confirming 
receipt of the new report is automatically sent to the 
sender. To ensure data quality on paper forms, every 



month members of the field support team visit each 
health centre to check registers against physician and 
patient records. All data from OPD and laboratory regis- 
ters are recorded in triplicate during the patient encoun- 
ter. During field support visits, members of the team 
conducting supervision extract one copy of the register 
pages; using a copy of the source data instead of transcrib- 
ing the source data removes one level of possible error. 
Where register data are incomplete, reconciliation is 
attempted using the routine physician patient records {i.e., 
OPD cards). Records are also cross-checked with the rou- 
tine laboratory and dispensary registers to confirm tests 
were performed and drugs dispensed, as indicated. 

At the team headquarters in Addis Ababa, all OPD 
and laboratory data are entered in duplicate into a .dbf 
IV file format to create an individual patient-level dataset 
for each facility. Within the SMS system, data are sub- 
mitted electronically and automatically entered into a 
seperate database. An external assessment of this system 
was conducted in April 2011, one year after implementa- 
tion (data not shown). To verify quality of data collected 
through SMS reports in an internal audit, weekly aggre- 
gate indicators were re-calculated using the primary 
paper registry data from all 83 sites for 2011 and com- 
pared to results from the SMS weekly aggregate reports, 
yielding 2,759 weekly reports from the SMS dataset and 
2,759 corresponding observations based on paper regis- 
tries. The paper data consisting of individual patient re- 
cords were used to re-calculate all weekly indicators by 
project staff and compared to the corresponding SMS 
reports at health centres and health posts using a Bland- 
Altman framework [19,20], wherein the difference be- 
tween two measurements (or reports) is plotted against 
the average of the two measurements so that both mag- 
nitude of error as well as trends in error with measure- 
ment size (bias), can be visualized and quantified. 
Measurement differences were also plotted by time. 
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While this analysis was conducted for all indicators pre- 
sented here are results on the number of confirmed 
P. falciparum cases reported only, as this is generally felt 
to be the most sensitive and important public health in- 
dicator collected by the system. The results indicate that 
overall concordance between the two reporting systems 
(paper recording and SMS reports) is high (Spearman's 
p was greater than 0.73 for all indicators with only one 
result showing a p less than 0.79), but that the size of 
error was likely to grow with the size of the actual 
weekly indicator reported, at least at very low ranges of 
the average of the indicators calculated using the SMS 
and the paper registry data (Figure 3). Only 80 facility 
reports of the 2,759 week reports (including health cen- 
tres and health posts) (2.9%) showed a difference be- 
tween SMS and paper reports of more than ±75 patients 
per week. A large fraction of these cases were due to 
missing SMS reports in high volume weeks based on the 
paper records. The concordance of SMS reporting with 
paper records appeared to improve after a relatively 
short 15-week "burn- in" period (Figure 4). When nor- 
malized to the average of the two reports, differences be- 
tween the two reporting systems were relatively larger 
when reporting small numbers for a specific indicator, 
but these fell to very low relative error sizes when 
reporting larger numbers. 

An External Quality Assurance (EQA) system for mal- 
aria microscopy was developed at the start of the project 
based on WHO guidelines to ensure that quality diag- 
nostics were being applied at the health centre level 
(i.e. where microscopy remains the standard method for 
laboratory confirmation of malaria infection). The sys- 
tem sampled ten slides per month (stratified to include 
five positive slide and five negative slides using system- 
atic random sampling from the laboratory register). 
Slides were re-read by an expert microscopist blinded to 
the facility result and re-read by a second expert micro- 
scopist centrally in the event that the gold standard 
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Figure 3 Bland-Altman plot of total outpatients seen at each 
facility (health centres and health posts) per week from the 
SMS system and the paper system during 201 1. The difference 
between the two measures is shown on the y-axis and the average 
of the two measures is shown on the x-axis. 
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Figure 4 Plot of differences between paper and SMS reports of 
confirmed Plasmodium falciparum cases per week over time at 
all facilities (health centres and health posts). 



reading was in disagreement with the facility result. Gold 
standard results were considered to be the two readings 
which were in agreement. Analysis of EQA data was re- 
ported back to facilities during monthly meetings com- 
bined with advice about the proper preparation and 
reading of slides based on an expert review of slide 
staining technique. Results of the EQA are presented in 
Figure 5. Overall concordance on both positivity and spe- 
cies identification were relatively high with >75% average 
for each facility, but imperfect with significant month to 
month variation. Logistic random effects regression ana- 
lysis showed that overall there was no significant trend in 
accuracy for malaria diagnosis over time across all ten 
health centres, but that performance varied significantly 
by facility; within facility trends only appeared statistically 
significant in two facilities (both were positive). One facil- 
ity performed especially poorly initially, but also showed 
the largest improvement over time. 

Malaria at sentinel sites 

Data on outpatient testing and laboratory confirmation 
of malaria cases has been collected in the sentinel sites 
over a period of 43 months. To examine trends in cases 
over time, multilevel time series Poisson random effects 
regression models were fit to data across all sites and 
over the entire time series with fixed effect covariates for 
month and year (either as a factor variable or as a linear 
time trend). Outcome variables were either all confirmed 
malaria cases or species-specific confirmed malaria cases 
(P. falciparum or P. vivax) with the exposure of total 
outpatients attending the facility during the month. Ran- 
dom intercepts were included for both the facility and 
PHCUs (i.e. health centres/satellite health posts). 

Data indicate a downward trend in overall confirmed 
malaria cases across sentinel sites between 2010 and 
2013, after adjusting for seasonality and the variation in 
incidence levels between individual facilities and across 
PHCU (Table 1). They also indicate that the trend was 
much more prominent in P. vivax than in P. falciparum 
cases. Linear trend models show approximately a 13% 
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Figure 5 Results of Laboratory External Quality Assurance for malaria microscopy: proportion of blood slides read correctly for 
presence of malaria parasites at health centres (black lines are linear trend lines). 



(95% CI 11.2 - 14.8) reduction per year of P. falciparum 
cases; treating year as a factor suggests that most of this 
decline resulted from a 27% reduction in 2012 compared 
to 2010 levels. Early data from 2013 indicate a return in 
P. falciparum case levels similar to that observed in 
2010. These results indicate that changes in P. falcip- 
arum cases at the sentinel sites most likely represent 



inter- annual variation rather than a general trend. Model 
results for P. falciparum alone are only presented with a 
linear trend for consistency across the three outcomes 
(P. falciparum, P. vivax and all malaria cases) but are 
best interpreted using the year factor model. These 
models do not adjust for rainfall or other possible 
confounders and, thus, make no effort to explain the 



Table 1 Regression modeled estimates of trends in malaria cases at all sentinel sites 


Model 


N (facility-month observations) 


Coefficient 


Incidence rate ratio 


Standard error 


p-value 


All cases linear trend 


1528 


Year (Linear trend) 


0.85 


.006 


<0.001 


All cases year factor 


1528 


Year 2010 


Ref 










Year 201 1 


1.04 


.014 


0.010 






Year 2012 


.71 


.011 


<0.001 






Year 2013 


.78 


.028 


<0.001 


P. falciparum cases linear trend 


1528 


Year 


.87 


.009 


<0.001 


P. falciparum cases year factor 


1528 


Year 2010 


Ref 










Year 201 1 


1.19 


.024 


<0.001 






Year 2012 


.74 


.017 


<0.001 






Year 2013 


.99 


.065 


0.860 


P. vivax cases linear trend 


1528 


Year (Linear trend) 


.80 


.008 


<0.001 


P. vivax cases year factor 


1528 


Year 2010 


Ref 










Year 201 1 


.85 


.017 


<0.001 






Year 2012 


.65 


.014 


<0.001 






Year 2013 


.46 


.023 


<0.001 



All regression models include month of the year as a factor variable, and random intercepts at both the primary health care unit level and the health facility level. 
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reasons behind trends, but only seek to accurately esti- 
mate the overall trend in malaria incidence in the senti- 
nel facilities. While full data for 2013 were not available 
at the time of manuscript submission, the inclusion of 
month in the models to account for seasonality provides 
the ability to estimate whether 2013 malaria incidence 
was higher or lower for the months with already avail- 
able data. Overall the incidence rate ratios indicate that 
across the sentinel sites there has been an approximately 
15% (95% CI 13.8%-16.2%) fall in overall incidence rates 
per year since 2010 (Table 1, Figures 6 and 7). By June 
2013, the sentinel surveillance system had identified 
three major malaria epidemics: a P. falciparum epidemic 
occurring in the catchment of Bulbulla Health Centre in 
June 2010; a P. vivax epidemic occurring in the catch- 
ment of Tulu Bollo Health Centre in October 2010; and 
a mixed but largely P. falciparum epidemic occurring in 
the catchment of Guangua Health Centre which lasted 
through the majority of 2011. Several smaller malaria 
case buildups were also noted by the system as well as 
several small outbreaks of relapsing fever. 



Figure 8 presents the proportion of laboratory cases 
and total outpatients seen at the health centre and 
health post level after full scale up of surveillance to all 
health posts. The results indicate that the health centres 
see the vast majority of all patients, typically greater than 
80% in any given month without much evidence of sea- 
sonal trends, while approximately half of the malaria 
burden based on confirmed cases is seen at the commu- 
nity level. As no control data is available it cannot be de- 
termined from these data whether this reflects a shift in 
treatment seeking from health centres to the community 
level after the development of the Health Extension Pro- 
gram, which resulted in the mass deployment of HEWs 
at the community level in Ethiopia, or an overall in- 
crease in treatment-seeking. Data from two malaria indi- 
cator surveys (2007 and 2011), however, documented a 
dramatic increase in the proportion of febrile children 
seeking treatment within 24 hours (15% to 51%) between 
the two surveys (MIS 2007 and 2011) [13,21]. During 
the same period (2007-2011) the number of health posts 
in Ethiopia increased by more than 150%, indicating that 
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Figure 6 Trends in confirmed malaria cases at all sentinel health centres from January 2010 to August, 2013. 
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the burden treated at the community (health post) level 
may indeed have been largely missed in surveillance sys- 
tems prior to the availability of community level treat- 
ment and surveillance. Only two malaria specific deaths 
were recorded in the health facilities during the period 
of observation. 

Discussion 

This sentinel surveillance system captures data related 
to malaria morbidity, mortality, diagnosis, and treat- 
ment. The discussion of this paper focuses on three key 
implementation challenges affecting data interpretation, 
as well as possible solutions. The three challenges in- 
clude: the data recording and reporting process, use of 
diagnostics, and data use for decision-making. 

Generating timely and accurate data 

Although the paper-based data reporting process at sen- 
tinel sites has been successful in delivering quality data 
from health centres and health posts to stakeholders on 
a monthly basis, the reliance on periodic field-based sup- 
port visits for data collection slowed the process and 
was resource-intensive. Additionally, rapid reporting be- 
came even more challenging when health post (commu- 
nity level) data were included. The integration of weekly 
SMS reporting greatly improved the timeliness of the 
surveillance system with relatively little loss of fidelity, 
especially after a "burn-in" period. Use of SMS and over- 
all maturation of the system allowed the scaling back of 
supervision from twice monthly to once monthly, thus 
improving the overall efficiency of the surveillance sys- 
tem. Over time, supervision could potentially be reduced 
to once every two or three months, assuming staff turn- 
over at health facilities is kept to a minimum. 

Capturing complete and accurate patient level data 

The sentinel surveillance system protocol requires that 
clinicians and laboratory staff enter data in their respect- 
ive registers at the time of the patient encounter (or ac- 
curately capture such data from patient cards at a later 
date). Laboratory registers in Ethiopia have remained 
consistent, reliably filled and comprehensive. However, 
OPD registers have at times been incomplete, particu- 
larly for identifying suspect malaria cases; a problem 
which will not be remedied by the use of SMS reporting 
systems. The OPD registers' incompleteness is further 
compounded by the use of multiple data forms, includ- 
ing an HMIS register that does not capture the neces- 
sary data for malaria surveillance (e.g. HMIS indicators 
currently capture neither testing rates nor consistently 
differentiate between laboratory confirmed and clinical 
malaria diagnoses). Failure to fully complete OPD regis- 
ters used by the surveillance sites may be complicated 
by a high reporting burden at the facility level for both 



malaria and non-malaria illnesses. This burden may in- 
clude in some locations newly expanded HMIS clinic 
registers, integrated disease surveillance and response 
(IDSR) reports and other HMIS and surveillance reports. 
Incomplete OPD registers may influence measures of 
appropriate testing of suspect malaria cases; records of a 
patients fever history or clinical suspicion of malaria are 
required to calculate testing rates, a key performance in- 
dicator for facilities scaling up universal diagnosis for 
suspect cases. Improved patient cards which integrate 
data collection needs of the HMIS with malaria surveil- 
lance data needs could possibly be used to remedy this 
problem. Furthermore, the role of the district health of- 
fice (or equivalent) in the system should be reinforced 
by establishing a focal point at the district office to serve 
as the liaison between surveillance systems and the 
health facilities. 

Challenges related to incomplete data at health facil- 
ities highlight the importance of collaboration and co- 
ordinating data collection across multiple branches of 
the health system. If the HMIS captured all of the key 
indicators needed to inform malaria control, parallel 
data collection systems for malaria surveillance might 
not be necessary. Clearly, an extra layer of enhanced mal- 
aria indicator data collection and reporting does poten- 
tially create a reporting burden: staff may be resistant to 
the additional reporting, and may resist participation in 
the data collection and reporting process, which in turn 
may contribute to reduced data quality and potentially 
make measurement of certain trends unreliable. Towards 
the end of the global malaria eradication efforts in 
Ethiopia, diagnosis and treatment for malaria was re- 
integrated into the overall health system and multiple 
reporting mechanisms existed for disease surveillance. 
Over time these came to include the HMIS, IDSR, public 
health emergency management (PHEM) and sentinel sur- 
veillance systems. Failure to harmonize data collection and 
indicators across systems leads to increased reporting bur- 
dens, and conflicting estimates of indicators [14]. 

In most locations, government-provided public health 
care is only one of the many possible options that pa- 
tients use to seek care for febrile illness, and malaria 
transmission and treatment-seeking patterns may fluctu- 
ate over time [22-24], Given these issues, surveillance 
systems must be able to account for changes in patient 
treatment-seeking behaviour and provider practices, and 
should expand the points of data collection from one 
type of public health facility to an assortment of pro- 
viders, including multiple health system levels in the 
same geographic areas, as well as the potential incorpor- 
ation of private providers if monitoring trends in mor- 
bidity and mortality are a focus of the system. 

The malaria diagnostic protocol for the sentinel sur- 
veillance system requires laboratory confirmation of all 
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suspect cases by either RDT or microscopy. Adherence 
to the diagnostic protocol was measured by calculating 
test uptake from the OPD and laboratory registers: the 
number of patients tested (from laboratory register) di- 
vided by the number of patients with suspect malaria 
(from the OPD register). The number of cases tested in 
some facility-month reports were far greater than the 
number of suspect cases registered through the OPD, 
resulting in reported test uptake over 100%, and suggest- 
ing the possibility that some patients are receiving la- 
boratory services for malaria without first registering, 
seeing a clinician, and being recorded in the reviewed 
OPD registers as a suspect case. Alternative explanations 
for this discrepancy include: incomplete registration of 
patients in OPD registers, referrals to the laboratory 
from service departments at health centres other than 
the OPD, and the potential for clinicians who use the 
malaria diagnostic test to rule out infection to refer pa- 
tients for testing who they do not consider as having 
suspected malaria. Unfortunately, matching data from 
OPD registers to laboratory registers has been challen- 
ging due to the frequent duplication of patient OPD 
card numbers. The challenges associated with calculat- 
ing test uptake using two separate and discrepant data 
sources makes this indicator of little use to the system. 
This indicator can be calculated only using OPD re- 
cords, but the implications in terms of bias of doing so 
are currently not known. 

Making data useful and generalizable 

One primary purpose of this sentinel surveillance system 
was to detect malaria outbreaks through regular analysis 
and reporting of data to inform appropriate public 
health responses. By June 2013, the sentinel surveillance 
system had identified three major malaria epidemics: a 
P. falciparum epidemic at Bulbula Health Centre in June 
2010, a P. vivax epidemic at Tulu Bollo Health Centre 
in October 2010, and a mixed but largely P. falcipa- 
rum epidemic at Guangua Health Centre which lasted 
through the majority of 2011. Standard responses to 
these epidemics were to notify local health authorities 
and malaria control partners, and to ensure that health 
centres and health posts were adequately supplied for 
testing and treatment; local partners and authorities also 
increased targeted communication interventions in the 
area, and in some cases targeted indoor residual spraying 
of households with insecticide was also used (in the 
Guangua epidemic). 

There are inherent challenges in applying standard 
epidemic detection methods depending upon the coun- 
try context. In the absence of high quality historical data 
on malaria incidence, application of methods such as the 
WHO monthly methods or standard process control 
techniques is challenging or inappropriate. For this 



reason the project has moved toward the use of spatio- 
temporal regression methods for detection of increased 
local incidence of malaria. Collecting information on 
place of residence may aid in the identification of spatial 
clustering of cases; however, the challenges in reconciling 
local place names when literacy levels in the population 
are low, standardized maps and place names do not exist, 
and discordant place names appear in multiple registers 
for the same patient should not be underestimated. 

Reliance on historical data or malaria microscopy may 
also be a poor indicator of malaria clinical incidence. At 
the health post level, multi-species malaria RDTs are uti- 
lized exclusively, and it currently appears that at least 
half of all malaria cases in Ethiopia are diagnosed based 
upon RDT results. Ethiopia has low malaria transmission 
levels as evidenced by the 1.3% microscopy slide preva- 
lence found in the Malaria Indicator Survey performed 
in 2011 [13,21]. When there is very low malaria trans- 
mission, false negativity due to low sensitivity of both 
RDTs and microscopy at low parasite densities may 
prove problematic [25-27]. Conversely, false positives 
may lead to overuse of anti-malarial drugs and a failure 
to properly delineate areas of the country absent of mal- 
aria risk. Malaria prevention and control programmes 
must ensure quality of laboratory confirmation especially 
in low transmission areas. 

Data from the sentinel surveillance system are also 
used for informing routine program decision making, 
such as monitoring commodity stocks and forecasting 
demand, estimating and forecasting the burden of mal- 
aria expected at health posts, health centres and hospi- 
tals. During support visits, staff review data and notify 
the district office and the health centre director of any 
substantial increase in malaria cases. Monthly summary 
reports from all ten sentinel districts are also shared 
with senior ORHB officials, senior PHEM officials and 
with the FMOH for their information and action. On 
several occasions reports of malaria case buildups from 
districts nearby the sentinel sites prompted queries from 
the FMOH as to whether similar trends could be con- 
firmed at nearby sentinel sites. Additionally, these offi- 
cials were able to check case summaries in real-time by 
health facility and health post as soon as they were 
posted online by weekly SMS malaria morbidity reports. 

While sentinel surveillance systems often lack the abil- 
ity to produce generalizable data beyond catchment 
areas, they can provide precise, accurate and real-time 
data from discrete areas that can assist in interpreting 
events that are in nearby districts and aggregated data 
from other districts. Such systems could be testing 
grounds for operational strategies to identify foci of in- 
creased malaria transmission or to pilot enhanced con- 
trol and elimination efforts, and provide an example of 
how to achieve high quality surveillance data in a 
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resource limited setting. Ethiopia's sentinel sites are cur- 
rently serving as grounds for testing strategies to expli- 
citly incorporate geo-spatial information on cases and 
travel histories into routinely collected facility-based 
data, as well as piloting the use of spatio-temporal re- 
gression analyses to identify locations with increased 
malaria risk to enhance the sensitivity and specificity of 
routine automated epidemic detection. 

Conclusions and recommendations 

This paper describes three implementation challenges that 
impacted a malaria surveillance systems performance: 1) 
ensuring a timely and accurate data reporting process; 2) 
capturing complete and accurate patient-level data; and 3) 
expanding the usefulness and generalizability of the sys- 
tems data to monitor progress towards the national mal- 
aria control goals of reducing malaria deaths and eventual 
elimination of transmission. 

Transmission of data more rapidly and accurately, and 
expanded dissemination of malaria surveillance data can 
be achieved using mobile phone technology. Such data 
can accurately represent registry data in the aggregate, at 
least in Oromia, and allows for tracking of trends in 
malaria in surveillance districts. The increased use of 
this technology across Africa offers promising opportun- 
ities to collect and disseminate surveillance data in a 
timely way. High quality malaria surveillance in Ethiopia 
remains a resource intensive activity and extending the 
generalizability of sentinel surveillance findings to other 
contexts remains a major limitation of these strategies. 
Future sentinel surveillance activities should continue to 
focus on expanding and ensuring quality control for 
and widespread use of diagnostics, delivering rapid data 
transmission with mobile technology, and enhancing 
generalizability and representativeness through wide ran- 
ging and appropriate site selection and inclusion of com- 
munity level testing and treatment in sentinel sites. 
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